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Introduction 


Zambia faces many challenges common to low- and middle-income countries as it seeks to improve its education 


system, including low literacy rates for young Zambians and low public expenditure on education. 


In the past 20 years, Zambia’s school system has expanded to reach more children in rural areas, primarily through 


community schools, which are autonomous from the government. But many community schools in Zambia are staffed 


by untrained and underpaid teachers who lack access to quality lesson plans, management skills, and basic supplies. 


These schools need a cost-effective solution for delivering quality education to improve learning outcomes. 


One potentially promising intervention for Zambia’s community schools is Impact Network’s eSchool 360 model, a 


multifaceted technology-aided instruction program designed to deliver low-cost education to children in rural areas. The 


model incorporates three potentially high-impact components: 


Standardized e-learning curriculum and pedagogy, through which schools receive technology—tablets and 
projectors powered through solar power—loaded with lesson plans for teachers and interactive lessons 


for students in the local language; 


Frequent teacher coaching and professional development on how to use the technology and enhance 


pedagogical skills; and 


Community ownership, meaning that teachers are recruited from the community and schools engage 


frequently with parents of students. 


RESEARCH OVERVIEW 


WHAT: AIR’s evaluation of a multifaceted technology-in- 
education program 


WHERE: Three districts in Zambia’s Eastern Province 
WHEN: Evaluation began in 2017 


HOW: Mixed-methods experimental study, with a cluster-randomized 
controlled trial across 63 schools and additional qualitative evidence- 
gathering methods, such as interviews and focus groups 


WHY: To contribute to the evidence base on use of technology in 
education, particularly its use in low-income rural areas, and to 
examine the feasibility of improving the quality of education by 
scaling the eSchool 360 model 


The American Institutes for Research (AIR) is conducting 
an evaluation of this model's effectiveness in support 
of its mission to use the best evidence available to 
improve people’s lives. This research will contribute 
to the body of existing evidence on the impact of 
technology on education in two important ways. 


1. Previous research shows that merely providing 
technology to schools is not enough to move the 
dial on student outcomes, and this evaluation 
examines the effect of technology as one 
component of a multifaceted intervention. 


2. The most rigorous evidence examining the use of 
technology in education in low- and middle-income 
countries comes from urban areas. This evaluation 


will provide evidence on the effect of technology- 
aided instruction programs on learning outcomes 
in rural areas of sub-Saharan Africa. 


ZAMBIA 


Methodology i 


To evaluate Impact Network’s eSchool 360 model, AIR designed and implemented 

a cluster-randomized controlled trial; 63 schools in three districts in poor, rural 

Zambia were assigned to either implement the model (30 schools) or serve as | 
control (33 schools). 


Fourteen months after the start of the program, researchers examined math, reading, oral vocabulary, and Zambian 
Achievement Test scores of children who lived in the areas of the 63 schools and were eligible to enroll in first grade 
when the program started. Researchers estimated the effects of: 


H The opportunity to enroll in a school that had implemented the model; and 


H Enrolling and actively attending a school that had implemented the model. 


The introduction of the model may have prompted some eligible children to enroll in schools with the model and thus 
changed enrollment and attendance, as well as the composition of the classroom. Because of these changes, children in 
schools with the eSchool 360 model are not comparable to children in other community schools. This comparison would 
produce invalid impact estimates. Estimating the effect of the opportunity to enroll in a school implementing the eSchool 
360 model allowed researchers to account for these changes in classroom composition when estimating how the model 
affected learning outcomes. 


Researchers also reviewed self-reported enrollment and attendance data and conducted additional investigations, including 
classroom observations, interviews, and focus groups, to understand the model’s implementation, how students and their 
parents experienced and perceived the model, and other potential effects of the model. 


A midline study, summarized here, focused on the impact and implementation of the Impact Network eSchool 360 model. 


Midline Findings Show Positive Effects of the eSchool 360 Model 
on Reading and Math Outcomes 


Although AIR’s evaluation of the Impact Network eSchool 360 model in Zambia is ongoing, midline results 
are promising: They show that well-designed, multifaceted, integrated technology-aided instruction programs 
can improve learning outcomes in the poorest areas of sub-Saharan Africa. 


Compared to children in the control group, children who Outcomes were strongest for students who attended a school 
had the opportunity to enroll in schools that implemented with the eSchool 360 model at least three times a week prior 
the eSchool 360 model had higher scores by: to the assessment. Specifically, these students had scores 


higher than comparable students in the control schools by: 
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= Students, parents, teachers, and program staff reported that they perceived schools with the eSchool 360 model to provide a higher 
quality of education, for example, because teacher attendance was higher. 


= Researchers found positive and statistically significant effects on parents’ perceptions of child engagement in school, and their 
satisfaction with school and teacher quality. 


= Compared to control areas, enrollment in schools in areas where the model was implemented was higher by 7.9 percentage points. 


= The model did not have a statistically significant effect on parents’ aspirations for their children’s education (measured by expectation 
that child will study beyond 12th grade). 


= In the rainy season, school attendance was lower, with some children working on their family farms, herding cattle (boys only), or caring 
for younger siblings. 


Although the midline results show the Impact Network eSchool 360 model had statistically significant effects on overall 
learning in the short-term, more research is needed to assess whether this model can achieve greater learning gains in 
the future. 


The next phase of AIR’s evaluation will examine program effects three years after the start of the program and whether 
the model is cost-effective. A previous study showed the model cost $3 per student per month, 70% less than what the 
Zambian government spent per student at the time of that study. AIR will update these results based on primary cost 
data collection. 
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